Image fusion method and bifocal camera

ABSTRACT

Embodiments of the present application are an image fusion method and a bifocal camera. The method includes: acquiring a thermal image captured by the thermal imaging lens and a visible light image captured by the visible light lens; determining a first focal length when the thermal imaging lens captures the thermal image and a second focal length when the visible light lens captures the visible light image; determining a size calibration parameter and a position calibration parameter of the thermal image according to the first focal length and the second focal length; adjusting a size of the thermal image according to the size calibration parameter, and moving the adjusted thermal image to the visible light image according to the position calibration parameter for registration with the visible light image, to obtain to-be-fused images; and fusing the to-be-fused images to generate a bifocal fused image.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a continuation of International ApplicationNo. PCT/CN2020/133974, filed on Dec. 4, 2020, which claims priority toChinese Patent Application No. 201911420526.4, filed on Dec. 31, 2019,which is incorporated herein by reference in its entirety.

TECHNICAL FIELD

Embodiments of the disclosure relate to the field of image processingtechnologies, and in particular, to an image fusion method and a bifocalcamera.

BACKGROUND

Thermal imaging and visible light imaging are two current widely usedimaging methods. In a desirable lighting situation, by visible lightimaging, a clear visible light image with rich details can be obtained.However, if the lighting situation is poor, the obtained visible lightimage is blurred and loses details. For example, in a strong lightenvironment, the visible light image obtained by visible light imaginghas an obvious halo and loses details. In a situation with lowvisibility such as a foggy day, the visible light image obtained byvisible light imaging is blurred. Thermal imaging is independent ofweather and light, and a clear thermal image can be obtained in anysituation by thermal imaging. However, the thermal image obtained bythermal imaging has no clear edge. Therefore, image fusion of thethermal image and the visible light image is provided to effectivelytake advantages of the two images, thereby capturing ideal images allthe time.

Currently, during the image fusion of the thermal image and the visiblelight image, feature point matching is performed on feature pointsextracted from the thermal image in the visible light image forregistration of the thermal image with the visible light image. Duringimplementation of the disclosure, the inventor finds the followingproblems. If focal lengths for capturing the thermal image and thevisible light image are different, imaging ratios of the thermal imageand the visible light image are different. In this case, during theregistration of the thermal image with the visible light image byfeature point matching of the feature points extracted from the thermalimage in the visible light image, the thermal image and the visiblelight image fail to coincide with each other. As a result, the thermalimage cannot be accurately registered with the visible light image,resulting in poor picture quality after fusion of the thermal image andthe visible light image.

SUMMARY

Embodiments of the disclosure aim to provide an image fusion method andapparatus, a bifocal camera, and an unmanned aerial vehicle (UAV). Byusing the embodiments of the disclosure, a thermal image can beaccurately registered with a visible light image when focal lengths forcapturing the thermal image and the visible light image are different,thereby improving picture quality of a fused image.

In order to resolve the foregoing technical problem, the embodiments ofthe disclosure adopt a technical solution that is as follows: An imagefusion method is provided. The method is applicable to a bifocal camera.The bifocal camera includes a thermal imaging lens and a visible lightlens. The method includes:

acquiring a thermal image captured by the thermal imaging lens and avisible light image captured by the visible light lens;

determining a first focal length when the thermal imaging lens capturesthe thermal image and a second focal length when the visible light lenscaptures the visible light image;

determining a size calibration parameter and a position calibrationparameter of the thermal image according to the first focal length andthe second focal length;

adjusting a size of the thermal image according to the size calibrationparameter, and moving the adjusted thermal image to the visible lightimage according to the position calibration parameter for registrationwith the visible light image, to obtain to-be-fused images; and

fusing the to-be-fused images to generate a bifocal fused image.

Optionally, the determining a size calibration parameter and a positioncalibration parameter of the thermal image according to the first focallength and the second focal length specifically includes:

determining a zoom scale of the thermal image according to the firstfocal length and the second focal length; and

determining the size calibration parameter and the position calibrationparameter of the thermal image according to the zoom scale.

Optionally, the thermal image includes a first rectangular calibrationframe, and the visible light image includes a second rectangularcalibration frame.

The determining a zoom scale of the thermal image according to the firstfocal length and the second focal length specifically includes:

determining first corner point coordinates of the first rectangularcalibration frame according to the first focal length, where the firstcorner point coordinates include first upper-left corner pointcoordinates, first lower-left corner point coordinates, firstupper-right corner point coordinates and first lower-right corner pointcoordinates;

determining second corner point coordinates of the second rectangularcalibration frame according to the second focal length, where the secondcorner point coordinates include second upper-left corner pointcoordinates, second lower-left corner point coordinates, secondupper-right corner point coordinates and second lower-right corner pointcoordinates; and

determining the zoom scale of the thermal image according to the firstcorner point coordinates and the second corner point coordinates.

Optionally, the determining the zoom scale of the thermal imageaccording to the first corner point coordinates and the second cornerpoint coordinates specifically includes:

determining a first width of the first rectangular calibration frameaccording to the first corner point coordinates;

determining a second width of the second rectangular calibration frameaccording to the second corner point coordinates; and

determining the zoom scale according to the first width and the secondwidth;

or

determining a first height of the first rectangular calibration frameaccording to the first corner point coordinates;

determining a second height of the second rectangular calibration frameaccording to the second corner point coordinates; and

determining the zoom scale according to the first height and the secondheight.

Optionally, the determining a first width of the first rectangularcalibration frame according to the first corner point coordinatesspecifically includes:

extracting any two corner point coordinates in the first corner pointcoordinates having a same ordinate, and determining the first widthaccording to a difference between abscissas of the extracted cornerpoint coordinates.

The determining a second width of the second rectangular calibrationframe according to the second corner point coordinates specificallyincludes:

extracting any two corner point coordinates in the second corner pointcoordinates having a same ordinate, and determining the second widthaccording to a difference between abscissas of the extracted cornerpoint coordinates.

The determining a first height of the first rectangular calibrationframe according to the first corner point coordinates specificallyincludes:

extracting any two corner point coordinates in the first corner pointcoordinates having a same abscissa, and determining the first heightaccording to a difference between ordinates of the extracted cornerpoint coordinates; and

The determining a second height of the second rectangular calibrationframe according to the second corner point coordinates specificallyincludes:

extracting any two corner point coordinates in the second corner pointcoordinates having a same abscissa, and determining the second heightaccording to a difference between ordinates of the extracted cornerpoint coordinates.

Optionally, the size calibration parameter includes a calibration widthand a calibration height, and the position calibration parameterincludes origin calibration coordinates.

The determining the size calibration parameter and the positioncalibration parameter of the thermal image according to the zoom scalespecifically includes:

acquiring a first initial width, a first initial height and firstcoordinates of a first positioning point that are of the thermal image;

determining second coordinates of a second positioning point in thevisible light image corresponding to the first positioning point;

determining the calibration width according to the zoom scale and thefirst initial width;

determining the calibration height according to the zoom scale and thefirst initial height; and

determining the origin calibration coordinates according to the zoomscale, the first coordinates and the second coordinates.

Optionally, the first coordinates are any of the first upper-left cornerpoint coordinates, the first lower-left corner point coordinates, thefirst upper-right corner point coordinates or the first lower-rightcorner point coordinates.

The second coordinates are any of the second upper-left corner pointcoordinates, the second lower-left corner point coordinates, the secondupper-right corner point coordinates or the second lower-right cornerpoint coordinates.

Optionally, after the step of fusing the to-be-fused images, the methodfurther includes:

determining a coincident region of a thermal image region and a visiblelight region in the to-be-fused images; and

cropping regions in the to-be-fused images other than the coincidentregion.

Optionally, the determining coincident region of a thermal image regionand a visible light region in the to-be-fused images specificallyincludes:

determining third corner point coordinates of the thermal image region,where the third corner point coordinates include third upper-left cornerpoint coordinates, third lower-left corner point coordinates, thirdupper-right corner point coordinates and third lower-right corner pointcoordinates;

determining fourth corner point coordinates of the visible light region,where the fourth corner point coordinates include fourth upper-leftcorner point coordinates, fourth lower-left corner point coordinates,fourth upper-right corner point coordinates and fourth lower-rightcorner point coordinates;

determining coordinates of a first corner point and a second cornerpoint of the coincident region according to the third corner pointcoordinates and the fourth corner point coordinates, where the firstcorner point and the second corner point are diagonal corner points; and

determining coordinates of a third corner point and a fourth cornerpoint of the coincident region according to the coordinates of the firstcorner point and the second corner point.

Optionally, an origin of the thermal image is any of an upper-leftcorner point, a lower-left corner point, an upper-right corner point ora lower-right corner point of the thermal image.

The determining third corner point coordinates of the thermal imageregion specifically includes:

determining the third corner point coordinates of the thermal imageregion according to the calibration width, the calibration height andthe origin calibration coordinates.

Optionally, an origin of the visible light image is any of an upper-leftcorner point, a lower-left corner point, an upper-right corner point ora lower-right corner point of the visible light image.

The determining fourth corner point coordinates of the visible lightregion specifically includes:

acquiring a second initial width, a second initial height and origincoordinates of the visible light image; and

determining the fourth corner point coordinates of the visible lightregion according to the second initial width, the second initial heightand the origin coordinates.

Optionally, the first corner point and the second corner point arerespectively a lower-left corner point and an upper-right corner pointof the coincident region.

Formulas for determining the coordinates of the first corner point andthe second corner point of the coincident region are as follows:

$\left\{ {\begin{matrix}{{x_{1}^{\prime} \leq x_{1}^{''} \leq x_{2}^{\prime}},{x_{1} = x_{1}^{''}}} \\{{x_{1}^{''} < x_{1}^{\prime}},{x_{1} = x_{1}^{\prime}}}\end{matrix},\left\{ \begin{matrix}{{y_{1}^{\prime} \leq y_{1}^{''} \leq y_{2}^{\prime}},{y_{1} = y_{1}^{''}}} \\{{y_{1}^{''} < y_{1}^{\prime}},{y_{1} = y_{1}^{\prime}}}\end{matrix} \right.} \right.$ $\left\{ {\begin{matrix}{{x_{2}^{''} \leq x_{2}^{\prime}},{x_{2} = x_{2}^{''}}} \\{{x_{2}^{''} > x_{2}^{\prime}},{x_{2} = x_{2}^{\prime}}}\end{matrix},\left\{ \begin{matrix}{{y_{2}^{''} \leq y_{2}^{\prime}},{y_{2} = y_{2}^{''}}} \\{{y_{2}^{''} > y_{2}^{\prime}},{y_{2} = y_{2}^{\prime}}}\end{matrix} \right.} \right.$

where (x₁,y₁) and (x₂′,y₂) are respectively the coordinates of the firstcorner point and the second corner point; (x₁″,y₂″) (x₁″,y₁″), (x₂″,y₂″)and (x₂″,y₁″) are respectively the third upper-left corner pointcoordinates, the third lower-left corner point coordinates, the thirdupper-right corner point coordinates and the third lower-right cornerpoint coordinates; and (x₁′,y₂′), (x₁′,y₁′), (x₂′,y₂′) and (x₂′,y₁′) arerespectively the fourth upper-left corner point coordinates, the fourthlower-left corner point coordinates, the fourth upper-right corner pointcoordinates and the fourth lower-right corner point coordinates.

In order to resolve the foregoing technical problem, the embodiments ofthe disclosure adopt another technical solution that is as follows: Animage fusion apparatus is provided. The apparatus is applicable to abifocal camera. The bifocal camera includes a thermal imaging lens and avisible light lens. The apparatus includes:

an acquisition module, configured to acquire a thermal image captured bythe thermal imaging lens and a visible light image captured by thevisible light lens;

a determination module, configured to: determine a first focal lengthwhen the thermal imaging lens captures the thermal image and a secondfocal length when the visible light lens captures the visible lightimage; and

determine a size calibration parameter and a position calibrationparameter of the thermal image according to the first focal length andthe second focal length;

an adjustment module, configured to adjust a size of the thermal imageaccording to the size calibration parameter, and move the adjustedthermal image to the visible light image according to the positioncalibration parameter for registration with the visible light image, toobtain to-be-fused images; and

a processing module, configured to fuse the to-be-fused images togenerate a bifocal fused image.

In order to resolve the foregoing technical problem, the embodiments ofthe disclosure adopt another technical solution that is as follows: Abifocal camera is provided, including:

a thermal imaging lens, configured to capture a thermal image;

a visible light lens, configured to capture a visible light image;

at least one processor; and

a memory, communicatively connected to the at least one processor.

The memory stores instructions executable by the at least one processor,the instructions, when executed by the at least one processor, causingthe at least one processor to perform the above image fusion method.

In order to resolve the foregoing technical problem, the embodiments ofthe disclosure adopt another technical solution that is as follows: AUAV is provided, including:

a fuselage;

an arm, connected to the fuselage;

a power apparatus, arranged on the arm; and

the above bifocal camera, where the bifocal camera is connected to thefuselage.

In order to resolve the foregoing technical problem, the embodiments ofthe disclosure adopt another technical solution that is as follows: Anon-volatile computer-readable storage medium is provided. Thenon-volatile computer-readable storage medium stores computer-executableinstructions used for causing a bifocal camera to perform the aboveimage fusion method.

In this embodiment of the disclosure, the size calibration parameter andthe position calibration parameter of the thermal image are determinedby using the first focal length during capture of the thermal image andthe second focal length during capture of the visible light image, thenthe size of the thermal image is adjusted according to the sizecalibration parameter, and the adjusted thermal image is moved to thevisible light image according to the position calibration parameter forregistration with the visible light image, to obtain the to-be-fusedimages, and the to-be-fused images are fused to generate the bifocalfused image. The size of the thermal image is adjusted according to thesize calibration parameter so that an imaging ratio of the thermal imageis the same as that of the visible light image. At this time, theadjusted thermal image is moved to the visible light image according tothe position calibration parameter for registration with the visiblelight image. Therefore, the thermal image can coincide with the visiblelight image, so that the thermal image is accurately registered with thevisible light image. In this way, picture quality of the fused image isimproved.

BRIEF DESCRIPTION OF THE DRAWINGS

One or more embodiments are exemplarily described with reference to thecorresponding figures in the accompanying drawings, and the descriptionsare not to be construed as limiting the embodiments. Components in theaccompanying drawings that have same reference numerals are representedas similar components, and unless otherwise particularly stated, thefigures in the accompanying drawings are not drawn to scale.

FIG. 1 is a schematic structural diagram of an unmanned aerial vehicle(UAV) according to an embodiment of the disclosure.

FIG. 2 is a schematic flowchart of an image fusion method according toan embodiment of the disclosure.

FIG. 3 is a schematic structural diagram of a thermal image and avisible light image according to an embodiment of the disclosure.

FIG. 4 is a schematic structural diagram of to-be-fused images accordingto an embodiment of the disclosure.

FIG. 5 is a schematic structural diagram of an image fusion apparatusaccording to an embodiment of the disclosure.

FIG. 6 is a schematic structural diagram of hardware of a bifocal cameraaccording to an embodiment of the disclosure.

DETAILED DESCRIPTION

To make the objectives, technical solutions and advantages of theembodiments of the disclosure clearer, the technical solutions in theembodiments of the disclosure will be described clearly and thoroughlywith reference to the accompanying drawings in the embodiments of thedisclosure. Apparently, the embodiments described are merely someembodiments, rather than all of the embodiments of the disclosure. Itshould be understood that the specific embodiments described herein aremerely used to explain the disclosure but are not intended to limit thedisclosure. All other embodiments obtained by a person of ordinary skillin the art based on the embodiments of the disclosure shall fall withinthe protection scope of the disclosure.

It should be noted that, when a component is expressed as “being fixedto” another component, the component may be directly on the anothercomponent, or one or more intermediate components may exist between thecomponent and the another component. When one component is expressed as“being connected to” another component, the component may be directlyconnected to the another component, or one or more intermediatecomponents may exist between the component and the another component.The terms “vertical”, “horizontal”, “left”, “right”, and similarexpressions in this specification are merely used for an illustrativepurpose.

In addition, technical features involved in the embodiments of thedisclosure described below may be combined with each other provided thatthere is no conflict between each other.

The disclosure provides an image fusion method and apparatus. The methodand the apparatus are applicable to a bifocal camera. By using themethod and apparatus, the bifocal camera can accurately register athermal image with a visible light image when focal lengths forcapturing the thermal image and the visible light image are different,thereby improving picture quality of a fused image. The bifocal cameraincludes a thermal imaging lens and a visible light lens. The thermalimaging lens is configured to capture a thermal image, and the visiblelight lens is configured to capture a visible light image.

The bifocal camera is applicable to a variety of robots, so that avariety of robots can capture high-quality images all the time. Therobots include but are not limited to an unmanned aerial vehicle (UAV),a sweeping robot or a navigation robot.

The disclosure is described in detail below by using an example in whichthe bifocal camera is applied to the UAV.

FIG. 1 shows a UAV 100 according to an embodiment of the disclosure. TheUAV includes a fuselage 10, an arm 20, a power apparatus 30, a bifocalcamera 40, a landing gear 50 and a flight control system. The arm 20,the bifocal camera 40 and the landing gear 50 are all connected to thefuselage 10. The flight control system is arranged in the fuselage 10.The power apparatus 30 is arranged on the arm 20. The power apparatus30, the bifocal camera 40 and the landing gear 50 are allcommunicatively connected to the flight control system. Therefore, theflight control system can control flight of the UAV 100 by using thepower apparatus 30, and can control the bifocal camera 40 to captureimages and control the landing gear 50 to unfold and fold. Aftercontrolling the bifocal camera 40 to capture images, the flight controlsystem can further acquire the captured images from the bifocal camera40.

In this embodiment of the disclosure, four arms 20 are arranged. Thefour arms are evenly distributed around the fuselage 10 and each areconfigured to carry the power apparatus 30.

The power apparatus 30 includes a motor and a propeller connected to ashaft of the motor. The motor can drive the propeller to rotate toprovide lift for the UAV 100 to realize flight. The motor can furtherchange a flight direction of the UAV 100 by changing a rotation speedand a rotation direction of the propeller. When the power apparatus 30is communicatively connected to the flight control system, the flightcontrol system can control the flight of the UAV 100 by controlling themotor.

The power apparatus 30 is arranged at an end of the arm 20 not connectedto the fuselage 10, and is connected to the arm 20 by the motor.

Preferably, the power apparatus 30 is arranged on each of the four armsof the UAV 100 to cause the UAV 100 to fly stably.

The bifocal camera 40 is arranged at a bottom of the fuselage 10 and cancapture images under the control of the flight control system.

The bifocal camera 40 includes a thermal imaging lens and a visiblelight lens. The thermal imaging lens is configured to capture a thermalimage, and the visible light lens is configured to capture a visiblelight image. When the bifocal camera 40 captures images, the thermalimaging lens and the visible light lens can simultaneously capture athermal image and a visible light image of a same object. Focal lengthswhen the thermal imaging lens and the visible light lens simultaneouslycapture the thermal image and the visible light image of the same objectmay be the same or different.

When the focal lengths of the thermal imaging lens and the visible lightlens are the same, imaging ratios of the thermal image and the visiblelight image are the same. When the focal lengths of the thermal imaginglens and the visible light lens are different, the imaging ratios of thethermal image and the visible light image are different. That is to say,the imaging ratios of thermal image and visible light image vary thefocal length.

Further, when capturing the thermal image, the thermal imaging lensperforms alignment to a target by using a first rectangular calibrationframe. The first rectangular calibration frame varies with the focallength of the thermal imaging lens, and is always aligned to the target.When capturing the visible light image, the visible light lens performsalignment to the target by using a second rectangular calibration frame.The second rectangular calibration frame varies with the focal length ofthe visible light lens, and is always aligned to the target.

When the bifocal camera 40 simultaneously captures the thermal image andthe visible light image of the same object by using the thermal imaginglens and the visible light lens, a size calibration parameter and aposition calibration parameter of the thermal image may be determined bydetermining a first focal length when the thermal imaging lens capturesthe thermal image and a second focal length when the visible light lenscaptures the visible light image, then a size of the thermal image isadjusted according to the size calibration parameter, and the adjustedthermal image is moved to the visible light image according to theposition calibration parameter for registration with the visible lightimage, to obtain to-be-fused images, and the to-be-fused images arefused to generate a bifocal fused image. The size of the thermal imageis adjusted according to the size calibration parameter so that animaging ratio of the thermal image is the same as that of the visiblelight image. At this time, the adjusted thermal image is moved to thevisible light image according to the position calibration parameter forregistration with the visible light image. Therefore, the thermal imagecan coincide with the visible light image, so that the thermal image isaccurately registered with the visible light image. In this way, picturequality of the fused image is improved.

In some embodiments, the bifocal camera 40 may be arranged on the bottomof the fuselage 10 by using a gimbal, to rotate with the gimbal tocapture images at different viewing angles.

The landing gear 50 is disposed on two opposite sides of the bottom ofthe fuselage 10, and is connected to the fuselage 10 by a drivingapparatus. The landing gear 50 may be driven by the driving apparatus tounfold and fold. When the UAV 100 is about to come into contact with theground, the driving apparatus controls the landing gear 50 to unfold.Therefore, the UAV 100 comes into contact with the ground by using thelanding gear 50. During the flight of the UAV 100, the driving apparatuscontrols the landing gear 50 to fold to prevent the landing gear 50 fromaffecting the flight of the UAV 100. When the landing gear 50 iscommunicatively connected to the flight control system, the flightcontrol system may control unfolding and folding of the landing gear 50by controlling the driving apparatus.

The flight control system is communicatively connected to the powerapparatus 30, the bifocal camera 40 and the landing gear 50 wiredly orwirelessly. The wireless connection includes but is not limited toWi-Fi, Bluetooth or ZigBee.

It may be understood that, in this embodiment of the disclosure, afterthe bifocal camera 40 fuses the thermal image and the visible lightimage by performing image fusion, even if the focal lengths forcapturing the thermal image and the visible light image are different,the thermal image can be accurately registered with the visible lightimage. In this way, a bifocal fused image having high picture quality isobtained. Therefore, the UAV 100 using the bifocal camera 40 can capturehigh-quality images all the time.

Further, FIG. 2 is a schematic flowchart of an image fusion methodaccording to an embodiment of the disclosure. The image fusion method isperformed by the above bifocal camera 40, to accurately register athermal image with a visible light image when focal lengths forcapturing the thermal image and the visible light image are different,thereby improving picture quality of a fused image.

Specifically, the image fusion method includes the following steps:

S100: Acquiring a thermal image captured by the thermal imaging lens anda visible light image captured by the visible light lens.

S200: Determining a first focal length when the thermal imaging lenscaptures the thermal image and a second focal length when the visiblelight lens captures the visible light image.

In this embodiment of the disclosure, the thermal image and the visiblelight image are images of a same target simultaneously captured by thethermal imaging lens and the visible light lens.

Since the focal lengths when the thermal imaging lens and the visiblelight lens capture the thermal image and the visible light image of thesame object may be different, after the thermal image captured by thethermal imaging lens is acquired, the first focal length when thethermal imaging lens captures the thermal image is determined, and afterthe visible light image captured by the visible light lens is acquired,the second focal length when the visible light lens captures the visiblelight image is acquired.

A focal length set when the thermal imaging lens captures the thermalimage is determined as the first focal length when the thermal imaginglens captures the thermal image. A focal length set when the visiblelight lens captures the visible light image is determined as the secondfocal length when the visible light lens captures the visible lightimage.

S300: Determining a size calibration parameter and a positioncalibration parameter of the thermal image according to the first focallength and the second focal length.

When the focal lengths of the thermal imaging lens and the visible lightlens are different, imaging ratios of the thermal image and the visiblelight image are different. In this case, if the thermal image isdirectly registered with the visible light image, the thermal imagecannot coincide with the visible light image. Therefore, a sizecalibration parameter and a position calibration parameter of thethermal image are required to be determined, so as to calibrate thethermal image according to the size calibration parameter and theposition calibration parameter to coincide with the visible light image.

An imaging scale of the thermal image varies with the first focallength, and an imaging scale of the visible light image varies with thesecond focal length. Therefore, in order to ensure that the sizecalibration parameter and the position calibration parameter of thethermal image can be used to calibrate the thermal image to coincidewith the visible light image, the size calibration parameter and theposition calibration parameter of the thermal image vary with theimaging scale of the thermal image and the imaging scale of the visiblelight image, that is, vary with the first focal length and the secondfocal length. Therefore, in this embodiment of the disclosure, the sizecalibration parameter and the position calibration parameter of thethermal image are determined according to the first focal length and thesecond focal length. In this way, the determined size calibrationparameter and position calibration parameter can be used to calibratethe thermal image to coincide with the visible light image.

The size calibration parameter is used for adjusting a size of thethermal image so that the imaging scale of the thermal image is the sameas that of the visible light image. The size calibration parameterincludes a calibration width and a calibration height. The calibrationwidth is used for adjusting a width of the thermal image. Thecalibration height is used for adjusting a height of the thermal image.

The position calibration parameter is used for adjusting a position ofthe thermal image in the visible light image, so that the thermal imageand the visible light image having the same imaging scale coincide witheach other. The position calibration parameter includes origincalibration coordinates. The origin calibration coordinates are used foradjusting an origin position of the thermal image.

In some embodiments, the determining a size calibration parameter and aposition calibration parameter of the thermal image according to thefirst focal length and the second focal length specifically includes:determining a zoom scale of the thermal image according to the firstfocal length and the second focal length; and determining the sizecalibration parameter and the position calibration parameter of thethermal image according to the zoom scale.

Referring to FIG. 3 , since the thermal imaging lens performs alignmentto a target by using a first rectangular calibration frame whencapturing the thermal image, the thermal image includes the firstrectangular calibration frame. The first rectangular calibration framevaries with the first focal length. Since the visible light lensperforms alignment to the target by using a second rectangularcalibration frame when capturing the visible light image, the visiblelight image includes the second rectangular calibration frame. Thesecond rectangular calibration frame varies with the second focallength.

Since the first rectangular calibration frame varies with the firstfocal length, a correspondence table of a preset first focal length andpreset first corner point coordinates is pre-established. For example, apreset first focal length A1 corresponds to preset first corner pointcoordinates J1, where the preset first corner point coordinates J1include preset first upper-left corner point coordinates J11, presetfirst lower-left corner point coordinates J12, preset first upper-rightcorner point coordinates J13 and preset first lower-right corner pointcoordinates J14; and a preset first focal length A2 corresponds topreset first corner point coordinates J2, where the preset first cornerpoint coordinates J2 include preset first upper-left corner pointcoordinates J21, preset first lower-left corner point coordinates J22,preset first upper-right corner point coordinates J23 and preset firstlower-right corner point coordinates J24.

Since the second rectangular calibration frame varies with the secondfocal length, a correspondence table of a preset second focal length andpreset second corner point coordinates is pre-established. For example,a preset second focal length B1 corresponds to preset second cornerpoint coordinates K1, where the preset second corner point coordinatesK1 include preset second upper-left corner point coordinates K11, presetsecond lower-left corner point coordinates K12, preset secondupper-right corner point coordinates K13 and preset second lower-rightcorner point coordinates K14; and a preset second focal length B2corresponds to preset second corner point coordinates K2, where thepreset second corner point coordinates K2 include preset secondupper-left corner point coordinates K21, preset second lower-left cornerpoint coordinates K22, preset second upper-right corner pointcoordinates K23 and preset second lower-right corner point coordinatesK24.

Based on the above, the determining a zoom scale of the thermal imageaccording to the first focal length and the second focal lengthspecifically includes: determining first corner point coordinates of thefirst rectangular calibration frame according to the first focal length;determining second corner point coordinates of the second rectangularcalibration frame according to the second focal length; and determiningthe zoom scale of the thermal image according to the first corner pointcoordinates and the second corner point coordinates.

The first corner point coordinates include first upper-left corner pointcoordinates, first lower-left corner point coordinates, firstupper-right corner point coordinates and first lower-right corner pointcoordinates.

The second corner point coordinates include second upper-left cornerpoint coordinates, second lower-left corner point coordinates, secondupper-right corner point coordinates and second lower-right corner pointcoordinates.

During the determination of the first corner point coordinates of thefirst rectangular calibration frame according to the first focal length,a preset first focal length matching the first focal length isdetermined from the pre-established correspondence table of the presetfirst focal length and the preset first corner point coordinates, andpreset first corner point coordinates corresponding to the preset firstfocal length matching the first focal length are determined as the firstcorner point coordinates of the first rectangular calibration frame. Forexample, when the first focal length is A1, it may be determined thatthe preset first focal length A1 in the pre-established correspondencetable of the preset first focal length and the preset first corner pointcoordinates matches the first focal length A1. Therefore, the presetfirst corner point coordinates J1 corresponding to the preset firstfocal length A1 are determined as the first corner point coordinates ofthe first rectangular calibration frame. In this case, the firstupper-left corner point coordinates are J11, the first lower-left cornerpoint coordinates are J12, the first upper-right corner pointcoordinates are J13, and the first lower-right corner point coordinatesare J14.

During the determination of the second corner point coordinates of thesecond rectangular calibration frame according to the second focallength, a preset second focal length matching the second focal length isdetermined from the pre-established correspondence table of the presetsecond focal length and the preset second corner point coordinates, andpreset second corner point coordinates corresponding to the presetsecond focal length matching the second focal length are determined asthe second corner point coordinates of the second rectangularcalibration frame. For example, when the second focal length is B1, itmay be determined that the preset second focal length B1 in thepre-established correspondence table of the preset second focal lengthand the preset second corner point coordinates matches the second focallength B1. Therefore, the preset second corner point coordinates K1corresponding to the preset second focal length B1 are determined as thesecond corner point coordinates of the second rectangular calibrationframe. In this case, the second upper-left corner point coordinates areK11, the second lower-left corner point coordinates are K12, the secondupper-right corner point coordinates are K13, and the second lower-rightcorner point coordinates are K14.

During the determination of the zoom scale of the thermal imageaccording to the first corner point coordinates and the second cornerpoint coordinates, a first width of the first rectangular calibrationframe is determined according to the first corner point coordinates, asecond width of the second rectangular calibration frame is determinedaccording to the second corner point coordinates, and then the zoomscale is determined according to the first width and the second width.

Specifically, during the determination of the first width of the firstrectangular calibration frame according to the first corner pointcoordinates, any two corner point coordinates in the first corner pointcoordinates having a same ordinate are extracted, and the first width isdetermined according to a difference between abscissas of the extractedcorner point coordinates. For example, the first corner pointcoordinates include the first upper-left corner point coordinates J11,the first lower-left corner point coordinates J12, the first upper-rightcorner point coordinates J13 and the first lower-right corner pointcoordinates J14, where J11 is (x₃,y₄), J12 is (x₃,y₃), J13 is (x₄,y₄),J14 is (x₄,y₃), J11 and J13 are the corner point coordinates having thesame ordinate, and J12 and J14 are also the corner point coordinateshaving the same ordinate. In this case, J11 and J13 are extracted, orJ12 and J14 are extracted, and then the first width is determinedaccording to the difference between the abscissas of the extractedcorner point coordinates. The first width is x₄−x₃.

During the determination of the second width of the second rectangularcalibration frame according to the second corner point coordinates, anytwo corner point coordinates in the second corner point coordinateshaving a same ordinate are extracted, and the second width is determinedaccording to a difference between abscissas of the extracted cornerpoint coordinates. For example, the second corner point coordinatesinclude the second upper-left corner point coordinates K11, the secondlower-left corner point coordinates K12, the second upper-right cornerpoint coordinates K13 and the second lower-right corner pointcoordinates K14, where K11 is (x₅,y₆), K12 is (x₅,y₅), K13 is (x₆,y₆),K14 is (x₆,y₅), K11 and K13 are the corner point coordinates having thesame ordinate, and K12 and K14 are also the corner point coordinateshaving the same ordinate. In this case, K11 and K13 are extracted, orK12 and K14 are extracted, and then the second width is determinedaccording to the difference between the abscissas of the extractedcorner point coordinates. The second width is x₆−x₅.

In this case, a ratio of the second width to the first width isdetermined as the zoom scale. The zoom scale is

$\frac{x_{6} - x_{5}}{x_{4} - x_{3}}.$

In some embodiments, during the determination of the zoom scale of thethermal image according to the first corner point coordinates and thesecond corner point coordinates, a first height of the first rectangularcalibration frame may be further determined according to the firstcorner point coordinates, a second height of the second rectangularcalibration frame may be further determined according to the secondcorner point coordinates, and then the zoom scale is determinedaccording to the first height and the second height.

Specifically, during the determination of the first height of the firstrectangular calibration frame according to the first corner pointcoordinates, any two corner point coordinates in the first corner pointcoordinates having a same abscissa are extracted, and the first heightis determined according to a difference between ordinates of theextracted corner point coordinates. For example, the first corner pointcoordinates include the first upper-left corner point coordinates J11,the first lower-left corner point coordinates J12, the first upper-rightcorner point coordinates J13 and the first lower-right corner pointcoordinates J14, where J11 is (x₃,y₄), J12 is (x₃,y₃), J13 is (x₄,y₄),0.114 is (x₄,y₃), J11 and J12 are the corner point coordinates havingthe same abscissa, and J13 and J14 are also the corner point coordinateshaving the same abscissa. In this case, J11 and J12 are extracted, orJ13 and J14 are extracted, and then the first height is determinedaccording to the difference between the ordinates of the extractedcorner point coordinates. The first height is y₄−y₃.

During the determination of the second height of the second rectangularcalibration frame according to the second corner point coordinates, anytwo corner point coordinates in the second corner point coordinateshaving a same abscissa are extracted, and the second height isdetermined according to a difference between ordinates of the extractedcorner point coordinates. For example, the second corner pointcoordinates include the second upper-left corner point coordinates K11,the second lower-left corner point coordinates K12, the secondupper-right corner point coordinates K13 and the second lower-rightcorner point coordinates K14, where K11 is (x₅,y₆), K12 is (x₅,y₅), K13is (x₆,y₆), K14 is (x₆,y₅), K11 and K12 are the corner point coordinateshaving the same abscissa, and K13 and K14 are also the corner pointcoordinates having the same abscissa. In this case, K11 and K12 areextracted, or K13 and K14 are extracted, and then the second height isdetermined according to the difference between the ordinates of theextracted corner point coordinates. The second height is y₆−y₅.

In this case, a ratio of the second height to the first height isdetermined as the zoom scale. The zoom scale is

$\frac{y_{6} - y_{5}}{y_{4} - y_{3}}.$

Further, the determining the size calibration parameter and the positioncalibration parameter of the thermal image according to the zoom scalespecifically includes: acquiring a first initial width, a first initialheight and first coordinates of a first positioning point that are ofthe thermal image; determining second coordinates of a secondpositioning point in the visible light image corresponding to the firstpositioning point; determining the calibration width according to thezoom scale and the first initial width; determining the calibrationheight according to the zoom scale and the first initial height; anddetermining the origin calibration coordinates according to the zoomscale, the first coordinates and the second coordinates.

The first initial width and the first initial height are a width and aheight of the thermal image before adjustment.

The first positioning point is any corner point of the first rectangularcalibration frame. In this case, the first coordinates of the firstpositioning point are any of the first upper-left corner pointcoordinates, the first lower-left corner point coordinates, the firstupper-right corner point coordinates or the first lower-right cornerpoint coordinates. For example, the first positioning point may be anupper-left corner point of the first rectangular calibration frame, alower-left corner point of the first rectangular calibration frame, anupper-right corner point of the first rectangular calibration frame or alower-right corner point of the first rectangular calibration frame.When the first positioning point is the upper-left corner point of thefirst rectangular calibration frame, the first coordinates are the firstupper-left corner point coordinates. When the first positioning point isthe lower-left corner point of the first rectangular calibration frame,the first coordinates are the first lower-left corner point coordinates.When the first positioning point is the upper-right corner point of thefirst rectangular calibration frame, the first coordinates are the firstupper-right corner point coordinates. When the first positioning pointis the lower-right corner point of the first rectangular calibrationframe, the first coordinates are the first lower-right corner pointcoordinates.

The second positioning point is any corner point of the secondrectangular calibration frame. In this case, the second coordinates ofthe second positioning point are any of the second upper-left cornerpoint coordinates, the second lower-left corner point coordinates, thesecond upper-right corner point coordinates or the second lower-rightcorner point coordinates. For example, the second positioning point maybe an upper-left corner point of the second rectangular calibrationframe, a lower-left corner point of the second rectangular calibrationframe, an upper-right corner point of the second rectangular calibrationframe or a lower-right corner point of the second rectangularcalibration frame. When the second positioning point is the upper-leftcorner point of the second rectangular calibration frame, the secondcoordinates are the second upper-left corner point coordinates. When thesecond positioning point is the lower-left corner point of the secondrectangular calibration frame, the second coordinates are the secondlower-left corner point coordinates. When the second positioning pointis the upper-right corner point of the second rectangular calibrationframe, the second coordinates are the second upper corner-right pointcoordinates. When the second positioning point is the lower-right cornerpoint of the second rectangular calibration frame, the secondcoordinates are the second lower-right corner point coordinates.

The first positioning point and the second positioning point are pointscorresponding to each other. That is to say, when the first positioningpoint is the upper-left corner point of the first rectangularcalibration frame, the second positioning point is the upper-left cornerpoint of the second rectangular calibration frame. When the firstpositioning point is the lower-left corner point of the firstrectangular calibration frame, the second positioning point is thelower-left corner point of the second rectangular calibration frame.When the first positioning point is the upper-right corner point of thefirst rectangular calibration frame, the second positioning point is theupper-right corner point of the second rectangular calibration frame.When the first positioning point is the lower-right corner point of thefirst rectangular calibration frame, the second positioning point is thelower-right corner point of the second rectangular calibration frame.

Based on the above, during the determination of the calibration widthaccording to the zoom scale and the first initial width, a product ofthe first initial width and the zoom scale is determined as thecalibration width. For example, if the zoom scale is X, the firstinitial width is Wr, and the calibration width is Wj,

$\frac{Wj}{Wr} = {X.}$

During the determination of the calibration height according to the zoomscale and the first initial height, a product of the first initialheight and the zoom scale is determined as the calibration height. Forexample, if the zoom scale is X, the first initial height is Hr, and thecalibration height is Hj,

$\frac{Hj}{Hr} = {X.}$

During the determination of the origin calibration coordinates accordingto the zoom scale, the first coordinates and the second coordinates, anabscissa of the origin calibration coordinates is determined accordingto the zoom scale, an abscissa of the first coordinates and an abscissaof the second coordinates, and an ordinate of the origin calibrationcoordinates is determined according to the zoom scale, an ordinate ofthe first coordinates and an ordinate of the second coordinates.

The abscissa of the origin calibration coordinates is equal to theabscissa of the second coordinates minus a product of the abscissa ofthe first coordinates and the zoom scale. The ordinate of the origincalibration coordinates is equal to the ordinate of the secondcoordinates minus a product of the ordinate of the first coordinates andthe zoom scale. For example, assuming that the zoom scale is X and thatthe upper-left corner point of the first rectangular calibration frameis the first positioning point, the first coordinates are J11, where J11is (x₃,y₄). In this case, the upper-left corner point of the secondrectangular calibration frame is the second positioning point, and thesecond coordinates are K11, where K11 is (x₅,y₆). If (x₀,y₀) is theorigin calibration coordinates,

${\frac{x_{5} - x_{0}}{x_{3} - 0} = X},{{{and}\frac{y_{6} - y_{0}}{y_{4} - 0}} = {X.}}$

In some embodiments, during the determination of the size calibrationparameter and the position calibration parameter of the thermal imageaccording to the zoom scale, a correspondence table of a preset zoomscale, a preset size calibration parameter and a preset positioncalibration parameter may be pre-established. As shown in Table 1, eachpreset zoom scale corresponds to one preset size calibration parameterand one preset position calibration parameter.

TABLE 1 Preset size Preset position calibration calibration Preset zoomscale parameter parameter X1 Wj1, Hj1 x₀₁, y₀₁ X2 Wj2, Hj2 x₀₂, y₀₂ X3Wj3, Hj3 x₀₃, y₀₃ X4 Wj4, Hj4 x₀₄, y₀₄

Based on the above, the determining the size calibration parameter andthe position calibration parameter of the thermal image according to thezoom scale specifically includes: determining a preset zoom scalematching the zoom scale from the pre-established correspondence table ofthe preset zoom scale, the preset size calibration parameter and thepreset position calibration parameter, determining a preset sizecalibration parameter corresponding to the preset zoom scale matchingthe zoom scale as the size calibration parameter of the thermal image,and determining a preset position calibration parameter corresponding tothe preset zoom scale matching the zoom scale as the positioncalibration parameter of the thermal image. For example, when the zoomscale is X1, it may be determined that a preset zoom scale X1 in thepre-established correspondence table of the preset zoom scale, thepreset size calibration parameter and the preset position calibrationparameter matches the zoom scale X1. Therefore, the preset sizecalibration parameter Wj1, Hj1 corresponding to the preset zoom scale X1is determined as the size calibration parameter of the thermal image,and the preset position calibration parameter x₀₁, y₀₁ corresponding tothe preset zoom scale X1 is determined as the position calibrationparameter of the thermal image.

It may be understood that, in some other embodiments, during thedetermination of the size calibration parameter and the positioncalibration parameter of the thermal image according to the first focallength and the second focal length, a correspondence table of a presetfirst focal length, a preset second focal length, a preset sizecalibration parameter and a preset position calibration parameter may bepre-established. As shown in Table 2, each preset first focal lengthcorresponds to at least two preset second focal lengths, and each of theat least two preset second focal lengths corresponds to one preset sizecalibration parameter and one preset position calibration parameter. Theat least two preset second focal lengths corresponding to each presetfirst focal length are the same.

TABLE 2 Preset size Preset position Preset first Preset secondcalibration calibration focal length focal length parameter parameterf10 f20 Wj1, Hj1 x₀₁, y₀₁ f21 Wj2, Hj2 x₀₂, y₀₂ f22 Wj3, Hj3 x₀₃, y₀₃f11 f20 Wj4, Hj4 x₀₄, y₀₄ f21 Wj5, Hj5 x₀₅, y₀₅ f22 Wj6, Hj6 x₀₆, y₀₆f12 f20 Wj7, Hj7 x₀₇, y₀₇ f21 Wj8, Hj8 x₀₈, y₀₈ f22 Wj9, Hj9 x₀₉, y₀₉

Based on the above, the determining a size calibration parameter and aposition calibration parameter of the thermal image according to thefirst focal length and the second focal length specifically includes:determining, from the pre-established correspondence table of the presetfirst focal length, the preset second focal length, the preset sizecalibration parameter and the preset position calibration parameter, apreset first focal length matching the first focal length as a targetfocal length; determining a preset second focal length matching thesecond focal length from preset second focal lengths corresponding tothe target focal length, determining a preset size calibration parametercorresponding to the preset second focal length, in the preset secondfocal lengths corresponding to the target focal length, matching thesecond focal length as the size calibration parameter of the thermalimage, and determining a preset position calibration parametercorresponding to the preset second focal length, in the preset secondfocal lengths corresponding to the target focal length, matching thesecond focal length as the position calibration parameter of the thermalimage. For example, when the first focal length is f10 and the secondfocal length is f20, it may be determined from the pre-establishedcorrespondence table of the preset first focal length, the preset secondfocal length, the preset size calibration parameters and the presetposition calibration parameters that the preset first focal length f10matches a first focal length f10. Therefore, the preset first focallength f10 is determined as the target focal length, and then a presetsecond focal length matching the second focal length f20 is determinedfrom preset second focal lengths f20, f21 and f22 corresponding to thepreset first focal length f10. Since the preset second focal length f20matches the second focal length f20, the preset size calibrationparameter Wj1, Hj1 corresponding to the second focal length f20 isdetermined as the size calibration parameter of the thermal image, andthe preset position calibration parameter x₀₁, y₀₁ corresponding to thesecond focal length f20 is determined as the position calibrationparameter of the thermal image.

S400: Adjusting a size of the thermal image according to the sizecalibration parameter, and moving the adjusted thermal image to thevisible light image according to the position calibration parameter forregistration with the visible light image, to obtain to-be-fused images.

Specifically, during the adjustment of the size of the thermal imageaccording to the size calibration parameter, the first initial width ofthe thermal image is adjusted to the calibration width, and the firstinitial height of the thermal image is adjusted to the calibrationheight. At this time, the imaging scale of the adjusted thermal image isthe same as that of the visible light image.

during the movement of the adjusted thermal image to the visible lightimage according to the position calibration parameters for registrationwith the visible light image, an origin of the adjusted thermal image ismoved to the origin calibration coordinates in the visible light image.At this time, the thermal image and the visible light image having thesame imaging ratio can coincide with each other. The thermal image andthe visible light image coinciding with each other are registered toobtain to-be-fused images.

In the to-be-fused images, the adjusted thermal image is superimposed onthe visible light image to form a thermal image region and a visiblelight region (shown in FIG. 4 ).

S500: Fusing the to-be-fused images to generate a bifocal fused image.

In some embodiments, after the to-be-fused images are fused, acoincident region of a thermal image region and a visible light regionis required to be determined, and then regions in the to-be-fused imagesother than the coincident region are cropped.

The determining coincident region of a thermal image region and avisible light region in the to-be-fused images specifically includes:determining third corner point coordinates of the thermal image region;determining fourth corner point coordinates of the visible light region;determining coordinates of a first corner point and a second cornerpoint of the coincident region according to the third corner pointcoordinates and the fourth corner point coordinates, where the firstcorner point and the second corner point are diagonal corner points; anddetermining coordinates of a third corner point and a fourth cornerpoint of the coincident region according to the coordinates of the firstcorner point and the second corner point.

The third corner point coordinates include third upper-left corner pointcoordinates, third lower-left corner point coordinates, thirdupper-right corner point coordinates and third lower-right corner pointcoordinates.

The fourth corner point coordinates include fourth upper-left cornerpoint coordinates, fourth lower-left corner point coordinates, fourthupper-right corner point coordinates and fourth lower-right corner pointcoordinates.

Since the thermal image region is a region formed by the adjustedthermal image, when the origin of the thermal image is any of anupper-left corner point, a lower-left corner point, an upper-rightcorner point or a lower-right corner point of the thermal image, thedetermining third corner point coordinates of the thermal image regionspecifically includes: determining the third corner point coordinates ofthe thermal image region according to the calibration width, thecalibration height and the origin calibration coordinates.

During the determination of the third corner point coordinates of thethermal image region according to the calibration width, the calibrationheight and the origin calibration coordinates, if the origin of thethermal image is the upper-left corner point, the origin calibrationcoordinates are determined as the third upper-left corner pointcoordinates. In this case, an abscissa of the third lower-left cornerpoint coordinates is the same as an abscissa of the third upper-leftcorner point coordinates, and an ordinate of the third lower-left cornerpoint coordinates is an ordinate of the third upper-left corner pointcoordinates minus the calibration height. An abscissa of the thirdupper-right corner point coordinates is the abscissa of the thirdupper-left corner point coordinates plus the calibration width, and anordinate of the third upper-right corner point coordinates is the sameas the ordinate of the third upper-left corner point coordinates. Anabscissa of the third lower-right corner point coordinates is theabscissa of the third upper-left corner point coordinates plus thecalibration width, and an ordinate of the third lower-right corner pointcoordinates is the ordinate of the third upper-left corner pointcoordinates minus the calibration height.

If the origin of the thermal image is the lower-left corner point, theorigin calibration coordinates are determined as the third lower-leftcorner point coordinates. In this case, the abscissa of the thirdupper-left corner point coordinates is the same as the abscissa of thethird lower-left corner point coordinates, and the ordinate of the thirdupper-left corner point coordinates is the ordinate of the thirdlower-left corner point coordinates plus the calibration height. Theabscissa of the third upper-right corner point coordinates is theabscissa of the third lower-left corner point coordinates plus thecalibration width, and the ordinate of the third upper-right cornerpoint coordinates is the ordinate of the third lower-left corner pointcoordinates plus the calibration height. The abscissa of the thirdlower-right corner point coordinates is the abscissa of the thirdlower-left corner point coordinates plus the calibration width, and theordinate of the third lower-right corner point coordinates is the sameas the ordinate of the third lower-left corner point coordinates.

If the origin of the thermal image is the upper-right corner point, theorigin calibration coordinates are determined as the third upper-rightcorner point coordinates. In this case, the abscissa of the thirdupper-left corner point coordinates is the abscissa of the thirdupper-right corner point coordinates minus the calibration width, andthe ordinate of the third upper-left corner point coordinates is thesame as the ordinate of the third upper-right corner point coordinates.The abscissa of the third lower-left corner point coordinates is theabscissa of the third upper-right corner point coordinates minus thecalibration width, and the ordinate of the third lower-left corner pointcoordinates is the ordinate of the third upper-right corner pointcoordinates minus the calibration height. The abscissa of the thirdlower-right corner point coordinates is the same as the abscissa of thethird upper-right corner point coordinates, and the ordinate of thethird lower-right corner point coordinates is the ordinate of the thirdupper-right corner point coordinates minus the calibration height.

If the origin of the thermal image is the lower-right corner point, theorigin calibration coordinates are determined as the third lower-rightcorner point coordinates. In this case, the abscissa of the thirdupper-left corner point coordinates is the abscissa of the thirdlower-right corner point coordinates minus the calibration width, andthe ordinate of the third upper-left corner point coordinates is theordinate of the third lower-right corner point coordinates plus thecalibration height. The abscissa of the third lower-left corner pointcoordinates is the abscissa of the third lower-right corner pointcoordinates minus the calibration width, and the ordinate of the thirdlower-left corner point coordinates is the same as the ordinate of thethird lower-right corner point coordinates. The abscissa of the thirdupper-right corner point coordinates is the same as the abscissa of thethird lower-right corner point coordinates, and the ordinate of thethird upper-right corner point coordinates is the ordinate of the thirdlower-right corner point coordinates plus the calibration height.

Since the visible light region is a region formed by the visible lightimage, when the origin of the visible light image is any of anupper-left corner point, a lower-left corner point, an upper-rightcorner point or a lower-right corner point of the visible light image,the determining fourth corner point coordinates of the visible lightregion specifically includes: acquiring a second initial width Wk, asecond initial height Hk and origin coordinates of the visible lightimage; and determining the fourth corner point coordinates of thevisible light region according to the second initial width Wk, thesecond initial height Hk and the origin coordinates.

During the determination of the fourth corner point coordinates of thevisible light region according to the second initial width, the secondinitial height and the origin coordinates, if the origin of the visiblelight image is the upper-left corner point, the origin coordinates aredetermined as the fourth upper-left corner point coordinates. In thiscase, an abscissa of the fourth lower-left corner point coordinates isthe same as an abscissa of the fourth upper-left corner pointcoordinates, and an ordinate of the fourth lower-left corner pointcoordinates is an ordinate of the fourth upper-left corner pointcoordinates minus the second initial height. An abscissa of the fourthupper-right corner point coordinates is the abscissa of the fourthupper-left corner point coordinates plus the second initial width, andan ordinate of the fourth upper-right corner point coordinates is thesame as the ordinate of the fourth upper-left corner point coordinates.An abscissa of the fourth lower-right corner point coordinates is theabscissa of the fourth upper-left corner point coordinates plus thesecond initial width, and an ordinate of the fourth lower-right cornerpoint coordinates is the ordinate of the fourth upper-left corner pointcoordinates minus the second initial height.

If the origin of the visible light image is the lower-left corner point,the origin coordinates are determined as the fourth lower-left cornerpoint coordinates. In this case, the abscissa of the fourth upper-leftcorner point coordinates is the same as the abscissa of the fourthlower-left corner point coordinates, and the ordinate of the fourthupper-left corner point coordinates is the ordinate of the fourthlower-left corner point coordinates plus the second initial height. Theabscissa of the fourth upper-right corner point coordinates is theabscissa of the fourth lower-left corner point coordinates plus thesecond initial width, and the ordinate of the fourth upper-right cornerpoint coordinates is the ordinate of the fourth lower-left corner pointcoordinates plus the second initial height. The abscissa of the fourthlower-right corner point coordinates is the abscissa of the fourthlower-left corner point coordinates plus the second initial width, andthe ordinate of the fourth lower-right corner point coordinates is thesame as the ordinate of the fourth lower-left corner point coordinates.

If the origin of the visible light image is the upper-right cornerpoint, the origin coordinates are determined as the fourth upper-rightcorner point coordinates. In this case, the abscissa of the fourthupper-left corner point coordinates is the abscissa of the fourthupper-right corner point coordinates minus the second initial width, andthe ordinate of the fourth upper-left corner point coordinates is thesame as the ordinate of the fourth upper-right corner point coordinates.The abscissa of the fourth lower-left corner point coordinates is theabscissa of the fourth upper-right corner point coordinates minus thesecond initial width, and the ordinate of the fourth lower-left cornerpoint coordinates is the ordinate of the fourth upper-right corner pointcoordinates minus the second initial height. The abscissa of the fourthlower-right corner point coordinates is the same as the abscissa of thefourth upper-right corner point coordinates, and the ordinate of thefourth lower-right corner point coordinates is the ordinate of thefourth upper-right corner point coordinates minus the second initialheight.

If the origin of the visible light image is the lower-right cornerpoint, the origin coordinates are determined as the fourth lower-rightcorner point coordinates. In this case, the abscissa of the fourthupper-left corner point coordinates is the abscissa of the fourthlower-right corner point coordinates minus the second initial width, andthe ordinate of the fourth upper-left corner point coordinates is theordinate of the fourth lower-right corner point coordinates plus thesecond initial height. The abscissa of the fourth lower-left cornerpoint coordinates is the abscissa of the fourth lower-right corner pointcoordinates minus the second initial width, and the ordinate of thefourth lower-left corner point coordinates is the same as the ordinateof the fourth lower-right corner point coordinates. The abscissa of thefourth upper-right corner point coordinates is the same as the abscissaof the fourth lower-right corner point coordinates, and the ordinate ofthe fourth upper-right corner point coordinates is the ordinate of thefourth lower-right corner point coordinates plus the second initialheight.

Since the first corner point and the second corner point are diagonalcorner points, the first corner point and the second corner point may berespectively the lower-left corner point and the upper-right cornerpoint of the coincident region, or may be respectively the upper-leftcorner point and the lower-right corner point of the coincident region.

When the first corner point and the second corner point are respectivelythe lower-left corner point and the upper-right corner point of thecoincident region, the formulas for determining the coordinates of thefirst corner point and the second corner point of the coincident regionare as follows:

$\left\{ {\begin{matrix}{{x_{1}^{\prime} \leq x_{1}^{''} \leq x_{2}^{\prime}},{x_{1} = x_{1}^{''}}} \\{{x_{1}^{''} < x_{1}^{\prime}},{x_{1} = x_{1}^{\prime}}}\end{matrix},\left\{ \begin{matrix}{{y_{1}^{\prime} \leq y_{1}^{''} \leq y_{2}^{\prime}},{y_{1} = y_{1}^{''}}} \\{{y_{1}^{''} < y_{1}^{\prime}},{y_{1} = y_{1}^{\prime}}}\end{matrix} \right.} \right.$ $\left\{ {\begin{matrix}{{x_{2}^{''} \leq x_{2}^{\prime}},{x_{2} = x_{2}^{''}}} \\{{x_{2}^{''} > x_{2}^{\prime}},{x_{2} = x_{2}^{\prime}}}\end{matrix},\left\{ \begin{matrix}{{y_{2}^{''} \leq y_{2}^{\prime}},{y_{2} = y_{2}^{''}}} \\{{y_{2}^{''} > y_{2}^{\prime}},{y_{2} = y_{2}^{\prime}}}\end{matrix} \right.} \right.$

where (x₁,y₁) and (x₂,y₂) are respectively the coordinates of the firstcorner point and the second corner point; (x₁″,y₂″) (x₁″,y₁″), (x₂″,y₂″)and (x₂″,y₁″) are respectively the third upper-left corner pointcoordinates, the third lower-left corner point coordinates, the thirdupper-right corner point coordinates and the third lower-right cornerpoint coordinates; and (x₁′,y₂′), (x₁′,y₁′), (x₂′,y₂′) and (x₂′,y₁′) arerespectively the fourth upper-left corner point coordinates, the fourthlower-left corner point coordinates, the fourth upper-right corner pointcoordinates and the fourth lower-right corner point coordinates.

In this case, during the determination of the coordinates of the thirdcorner point and the fourth corner point of the coincident regionaccording to the coordinates of the first corner point and the secondcorner point, if the third corner point and the fourth corner point arerespectively an upper-left corner point and a lower-right corner pointof the coincident region, an abscissa of the first corner point isdetermined as an abscissa of the third corner point, and an ordinate ofthe second corner point is determined as an ordinate of the third cornerpoint, an abscissa of the second corner point is determined as anabscissa of the fourth corner point, and an ordinate of the first cornerpoint is determined as an ordinate of the fourth corner point.

A principle of a formula for determining coordinates of the first cornerpoint and the second corner point of the coincident region when thefirst corner point and the second corner point are respectively theupper-left corner point and the lower-right corner point of thecoincident region and a principle of a formula for determiningcoordinates of the first corner point and the second corner point of thecoincident region when the first corner point and the second cornerpoint are respectively a lower-left corner point and an upper-rightcorner point of the coincident region are the same as the above.Therefore, details are not described herein again.

In this embodiment of the disclosure, the size calibration parameter andthe position calibration parameter of the thermal image are determinedby using the first focal length during capture of the thermal image andthe second focal length during capture of the visible light image, thenthe size of the thermal image is adjusted according to the sizecalibration parameter, and the adjusted thermal image is moved to thevisible light image according to the position calibration parameter forregistration with the visible light image, to obtain the to-be-fusedimages, and the to-be-fused images are fused to generate the bifocalfused image. The size of the thermal image is adjusted according to thesize calibration parameter so that an imaging ratio of the thermal imageis the same as that of the visible light image. At this time, theadjusted thermal image is moved to the visible light image according tothe position calibration parameter for registration with the visiblelight image. Therefore, the thermal image can coincide with the visiblelight image, so that the thermal image is accurately registered with thevisible light image. In this way, picture quality of the fused image isimproved.

Further, FIG. 5 is a schematic structural diagram of an image fusionapparatus according to an embodiment of the disclosure. The functions ofthe modules of the image fusion apparatus are performed by the abovebifocal camera 40, to accurately register a thermal image with a visiblelight image when focal lengths for capturing the thermal image and thevisible light image are different, thereby improving picture quality ofa fused image.

It should be noted that, the term “module” used in this embodiment ofthe disclosure may refer to a combination of software and/or hardwareimplementing a predetermined function. Although the apparatus describedin the following embodiments may be implemented by using software, it isalso conceivable that the apparatus may be implemented by usinghardware, or a combination of software and hardware.

Specifically, the image fusion apparatus includes:

an acquisition module 200, configured to acquire a thermal imagecaptured by the thermal imaging lens and a visible light image capturedby the visible light lens;

a determination module 300, configured to: determine a first focallength when the thermal imaging lens captures the thermal image and asecond focal length when the visible light lens captures the visiblelight image; and

determine a size calibration parameter and a position calibrationparameter of the thermal image according to the first focal length andthe second focal length;

an adjustment module 400, configured to adjust a size of the thermalimage according to the size calibration parameter, and move the adjustedthermal image to the visible light image according to the positioncalibration parameter for registration with the visible light image, toobtain to-be-fused images; and

a processing module 500, configured to fuse the to-be-fused images togenerate a bifocal fused image.

In some embodiments, the determination module 300 is further configuredto:

determine a zoom scale of the thermal image according to the first focallength and the second focal length; and

determine the size calibration parameter and the position calibrationparameter of the thermal image according to the zoom scale.

In some embodiments, the thermal image includes a first rectangularcalibration frame, and the visible light image includes a secondrectangular calibration frame.

The determination module 300 is further configured to:

determine first corner point coordinates of the first rectangularcalibration frame according to the first focal length, where the firstcorner point coordinates include first upper-left corner pointcoordinates, first lower-left corner point coordinates, firstupper-right corner point coordinates and first lower-right corner pointcoordinates;

determine second corner point coordinates of the second rectangularcalibration frame according to the second focal length, where the secondcorner point coordinates include second upper-left corner pointcoordinates, second lower-left corner point coordinates, secondupper-right corner point coordinates and second lower-right corner pointcoordinates; and

determine the zoom scale of the thermal image according to the firstcorner point coordinates and the second corner point coordinates.

In some embodiments, the determination module 300 is further configuredto:

determine a first width of the first rectangular calibration frameaccording to the first corner point coordinates;

determine a second width of the second rectangular calibration frameaccording to the second corner point coordinates; and

determine the zoom scale according to the first width and the secondwidth;

or

determine a first height of the first rectangular calibration frameaccording to the first corner point coordinates;

determine a second height of the second rectangular calibration frameaccording to the second corner point coordinates; and

determine the zoom scale according to the first height and the secondheight.

In some embodiments, the determination module 300 is further configuredto:

extract any two corner point coordinates in the first corner pointcoordinates having a same ordinate, and determine the first widthaccording to a difference between abscissas of the extracted cornerpoint coordinates.

extract any two corner point coordinates in the second corner pointcoordinates having a same ordinate, and determine the second widthaccording to a difference between abscissas of the extracted cornerpoint coordinates.

extract any two corner point coordinates in the first corner pointcoordinates having a same abscissa, and determine the first heightaccording to a difference between ordinates of the extracted cornerpoint coordinates; and

extract any two corner point coordinates in the second corner pointcoordinates having a same abscissa, and determine the second heightaccording to a difference between ordinates of the extracted cornerpoint coordinates.

In some embodiments, the size calibration parameter includes acalibration width and a calibration height, and the position calibrationparameter includes origin calibration coordinates;

The determination module 300 is further configured to:

acquire a first initial width, a first initial height and firstcoordinates of a first positioning point that are of the thermal image;

determine second coordinates of a second positioning point in thevisible light image corresponding to the first positioning point;

determine the calibration width according to the zoom scale and thefirst initial width;

determine the calibration height according to the zoom scale and thefirst initial height; and

determine the origin calibration coordinates according to the zoomscale, the first coordinates and the second coordinates.

In some embodiments, the first coordinates are any of the firstupper-left corner point coordinates, the first lower-left corner pointcoordinates, the first upper-right corner point coordinates or the firstlower-right corner point coordinates; and

the second coordinates are any of the second upper-left corner pointcoordinates, the second lower-left corner point coordinates, the secondupper-right corner point coordinates or the second lower-right cornerpoint coordinates.

In some embodiments, the processing module 500 is further configured to:

determine a coincident region of a thermal image region and a visiblelight region in the to-be-fused images; and

crop regions in the to-be-fused images other than the coincident region.

In some embodiments, the processing module 500 is further configured to:

determine third corner point coordinates of the thermal image region,where the third corner point coordinates include third upper-left cornerpoint coordinates, third lower-left corner point coordinates, thirdupper-right corner point coordinates and third lower-right corner pointcoordinates;

determine fourth corner point coordinates of the visible light region,where the fourth corner point coordinates include fourth upper-leftcorner point coordinates, fourth lower-left corner point coordinates,fourth upper-right corner point coordinates and fourth lower-rightcorner point coordinates;

determine coordinates of a first corner point and a second corner pointof the coincident region according to the third corner point coordinatesand the fourth corner point coordinates, where the first corner pointand the second corner point are diagonal corner points; and

determine coordinates of a third corner point and a fourth corner pointof the coincident region according to the coordinates of the firstcorner point and the second corner point.

In some embodiments, an origin of the thermal image is any of anupper-left corner point, a lower-left corner point, an upper-rightcorner point or a lower-right corner point of the thermal image.

The processing module 500 is further configured to:

determine the third corner point coordinates of the thermal image regionaccording to the calibration width, the calibration height and theorigin calibration coordinates.

In some embodiments, an origin of the visible light image is any of anupper-left corner point, a lower-left corner point, an upper-rightcorner point or a lower-right corner point of the visible light image.

The processing module 500 is further configured to:

acquire a second initial width, a second initial height and origincoordinates of the visible light image; and

determine the fourth corner point coordinates of the visible lightregion according to the second initial width, the second initial heightand the origin coordinates.

In some embodiments, the first corner point and the second corner pointare respectively a lower-left corner point and an upper-right cornerpoint of the coincident region.

Formulas for determining the coordinates of the first corner point andthe second corner point of the coincident region are as follows:

$\left\{ {\begin{matrix}{{x_{1}^{\prime} \leq x_{1}^{''} \leq x_{2}^{\prime}},{x_{1} = x_{1}^{''}}} \\{{x_{1}^{''} < x_{1}^{\prime}},{x_{1} = x_{1}^{\prime}}}\end{matrix},\left\{ \begin{matrix}{{y_{1}^{\prime} \leq y_{1}^{''} \leq y_{2}^{\prime}},{y_{1} = y_{1}^{''}}} \\{{y_{1}^{''} < y_{1}^{\prime}},{y_{1} = y_{1}^{\prime}}}\end{matrix} \right.} \right.$ $\left\{ {\begin{matrix}{{x_{2}^{''} \leq x_{2}^{\prime}},{x_{2} = x_{2}^{''}}} \\{{x_{2}^{''} > x_{2}^{\prime}},{x_{2} = x_{2}^{\prime}}}\end{matrix},\left\{ \begin{matrix}{{y_{2}^{''} \leq y_{2}^{\prime}},{y_{2} = y_{2}^{''}}} \\{{y_{2}^{''} > y_{2}^{\prime}},{y_{2} = y_{2}^{\prime}}}\end{matrix} \right.} \right.$

where (x₁,y₁) and (x₂,y₂) are respectively the coordinates of the firstcorner point and the second corner point; (x₁″,y₁″) (x₁″,y₁″), (x₂″,y₂″)and (x₂″,y₁″) are respectively the third upper-left corner pointcoordinates, the third lower-left corner point coordinates, the thirdupper-right corner point coordinates and the third lower-right cornerpoint coordinates; and (x₁′,y₂′), (x₁′,y₁′), (x₂′,y₂′) and (x₂′,y₁′) arerespectively the fourth upper-left corner point coordinates, the fourthlower-left corner point coordinates, the fourth upper-right corner pointcoordinates and the fourth lower-right corner point coordinates.

The apparatus embodiment and the method embodiment are based on the sameconcept. Therefore, for the content of the apparatus embodiment,reference may be made to the method embodiment without mutual conflictamong content, and details are not described herein again.

In some other alternative embodiments, the acquisition module 200, thedetermination module 300, the adjustment module 400 and the processingmodule 500 may be processing chips of the bifocal camera 40.

In this embodiment of the disclosure, the size calibration parameter andthe position calibration parameter of the thermal image are determinedby using the first focal length during capture of the thermal image andthe second focal length during capture of the visible light image, thenthe size of the thermal image is adjusted according to the sizecalibration parameter, and the adjusted thermal image is moved to thevisible light image according to the position calibration parameter forregistration with the visible light image, to obtain the to-be-fusedimages, and the to-be-fused images are fused to generate the bifocalfused image. The size of the thermal image is adjusted according to thesize calibration parameter so that an imaging ratio of the thermal imageis the same as that of the visible light image. At this time, theadjusted thermal image is moved to the visible light image according tothe position calibration parameter for registration with the visiblelight image. Therefore, the thermal image can coincide with the visiblelight image, so that the thermal image is accurately registered with thevisible light image. In this way, picture quality of the fused image isimproved.

Further, FIG. 6 is a schematic structural diagram of hardware of abifocal camera according to an embodiment of the disclosure. The bifocalcamera includes

one or more processors 41 and a memory 42. In FIG. 6 , the bifocalcamera includes one processor 41, for example.

The processor 41 and the memory 42 may be connected by a bus or in othermanners. In FIG. 6 , the processor and the memory are connected by abus, for example.

As a non-volatile computer-readable storage medium, the memory 42 may beconfigured to store a non-volatile software program, a non-volatilecomputer-executable program and a module, for example, programinstructions corresponding to the image fusion method and the modules(for example, the acquisition module 200, the determination module 300,the adjustment module 400, the processing module 500 and the like)corresponding to the image fusion apparatus in the above embodiments ofthe disclosure. The processor 41 executes various functionalapplications and data processing of the image fusion method by executinga non-volatile software program, an instruction and a module stored inthe memory 42, that is, implements the image fusion method in the abovemethod embodiment and the functions of the modules of the aboveapparatus embodiment.

The memory 42 may include a program storage area and a data storagearea. The program storage area may store an operating system and anapplication program required for at least one function. The data storagearea may store data created according to the use of the image fusionapparatus and the like.

The storage data area further stores preset data, including acorrespondence table of a preset first focal length and preset firstcorner point coordinates, a correspondence table of a preset secondfocal length and preset second corner point coordinates, acorrespondence table of a preset zoom scale, a preset size calibrationparameter and a preset position calibration parameter and acorrespondence table of a preset first focal length, a preset secondfocal length, a preset size calibration parameter and a preset positioncalibration parameter.

In addition, the memory 42 may include a high-speed random accessmemory, and may also include a nonvolatile memory, for example, at leastone magnetic disk storage device, a flash memory, or anothernon-volatile solid-state storage device. In some embodiments, the memory42 optionally includes memories disposed remote to the processor 41, andthese remote memories may be connected to the processor 41 through anetwork. The foregoing examples of the network include, but are notlimited to, the Internet, an intranet, a local area network, a mobilecommunication network, and a combination thereof.

The program instructions and one or more modules are stored in thememory 42, and the program instructions and the one or more modules,when executed by the one or more processors 41, perform steps of theimage fusion method in any of the foregoing method embodiments, orimplement functions of the modules of the image fusion apparatus in anyof the foregoing apparatus embodiments.

For the foregoing product, the method provided in the embodiments of thedisclosure may be performed, and the corresponding functional modulesfor performing the method and beneficial effects thereof are provided.For technical details not described in detail in this embodiment,reference may be made to the method provided in the foregoingembodiments of the disclosure.

An embodiment of the disclosure further provides a non-volatilecomputer-readable storage medium, storing computer-executableinstructions. The computer-executable instructions, when executed by oneor more processors such as one processor 41 in FIG. 6 , may cause acomputer to perform steps of the image fusion method in any of theforegoing method embodiments, or implement functions of the modules ofthe image fusion apparatus in any of the foregoing apparatusembodiments.

An embodiment of the disclosure further provides a computer programproduct, including a computer program stored on a non-volatilecomputer-readable storage medium. The computer program includes programinstructions, and the program instructions, when executed by one or moreprocessors such as one processor 41 in FIG. 6 , may cause a computer toperform steps of the image fusion method in any of the foregoing methodembodiments, or implement functions of the modules of the image fusionapparatus in any of the foregoing apparatus embodiments.

The described apparatus embodiment is merely an example. The modulesdescribed as separate parts may or may not be physically separated, andparts displayed as modules may or may not be physical units, may belocated in one position, or may be distributed on a plurality of networkunits. A part or all of the modules may be selected according to actualrequirements to achieve the objectives of the solutions of theembodiments of the disclosure.

Through the description of the foregoing embodiments, a person skilledin the art may clearly understand that the embodiments may beimplemented by software in combination with a universal hardwareplatform, and may certainly be implemented by hardware. A person ofordinary skill in the art may understand that all or some of theprocesses of the methods in the foregoing embodiments may be implementedby a computer program instructing relevant hardware. The program may bestored in a computer-readable storage medium. During execution of theprogram, processes of the foregoing method embodiments may be included.The foregoing storage medium may be a magnetic disk, an optical disc, aread-only memory (ROM), a random access memory (RAM) or the like.

The foregoing descriptions are embodiments of the disclosure, and theprotection scope of the disclosure is not limited thereto. Allequivalent structure or process changes made according to the content ofthis specification and accompanying drawings in the disclosure or bydirectly or indirectly applying the disclosure in other relatedtechnical fields shall fall within the protection scope of thedisclosure.

Finally, it should be noted that: the foregoing embodiments are merelyused for describing the technical solutions of the disclosure, but arenot intended to limit the disclosure. Under the ideas of the disclosure,the technical features in the foregoing embodiments or differentembodiments may also be combined, the steps may be performed in anyorder, and many other changes of different aspects of the disclosurealso exists as described above, and these changes are not provided indetail for simplicity. Although the disclosure is described in detailwith reference to the foregoing embodiments, it should be appreciated bya person skilled in the art that, modifications may still be made to thetechnical solutions described in the foregoing embodiments, orequivalent replacements may be made to the part of the technicalfeatures; and these modifications or replacements will not cause theessence of corresponding technical solutions to depart from the scope ofthe technical solutions in the embodiments of the disclosure.

What is claimed is:
 1. An image fusion method, applicable to a bifocalcamera, the bifocal camera comprising a thermal imaging lens and avisible light lens, wherein the method comprises: acquiring a thermalimage captured by the thermal imaging lens and a visible light imagecaptured by the visible light lens; determining a first focal lengthwhen the thermal imaging lens captures the thermal image and a secondfocal length when the visible light lens captures the visible lightimage; determining a size calibration parameter and a positioncalibration parameter of the thermal image according to the first focallength and the second focal length; adjusting a size of the thermalimage according to the size calibration parameter, and moving theadjusted thermal image to the visible light image according to theposition calibration parameter for registration with the visible lightimage, to obtain to-be-fused images; and fusing the to-be-fused imagesto generate a bifocal fused image.
 2. The method according to claim 1,wherein the determining a size calibration parameter and a positioncalibration parameter of the thermal image according to the first focallength and the second focal length specifically comprises: determining azoom scale of the thermal image according to the first focal length andthe second focal length; and determining the size calibration parameterand the position calibration parameter of the thermal image according tothe zoom scale.
 3. The method according to claim 2, wherein the thermalimage comprises a first rectangular calibration frame, and the visiblelight image comprises a second rectangular calibration frame; and thedetermining a zoom scale of the thermal image according to the firstfocal length and the second focal length specifically comprises:determining first corner point coordinates of the first rectangularcalibration frame according to the first focal length, wherein the firstcorner point coordinates comprise first upper-left corner pointcoordinates, first lower-left corner point coordinates, firstupper-right corner point coordinates and first lower-right corner pointcoordinates; determining second corner point coordinates of the secondrectangular calibration frame according to the second focal length,wherein the second corner point coordinates comprise second upper-leftcorner point coordinates, second lower-left corner point coordinates,second upper-right corner point coordinates and second lower-rightcorner point coordinates; and determining the zoom scale of the thermalimage according to the first corner point coordinates and the secondcorner point coordinates.
 4. The method according to claim 3, whereinthe determining the zoom scale of the thermal image according to thefirst corner point coordinates and the second corner point coordinatesspecifically comprises: determining a first width of the firstrectangular calibration frame according to the first corner pointcoordinates; determining a second width of the second rectangularcalibration frame according to the second corner point coordinates; anddetermining the zoom scale according to the first width and the secondwidth.
 5. The method according to claim 4, wherein the determining afirst width of the first rectangular calibration frame according to thefirst corner point coordinates specifically comprises: extracting anytwo corner point coordinates in the first corner point coordinateshaving a same ordinate, and determining the first width according to adifference between abscissas of the extracted corner point coordinates;the determining a second width of the second rectangular calibrationframe according to the second corner point coordinates specificallycomprises: extracting any two corner point coordinates in the secondcorner point coordinates having a same ordinate, and determining thesecond width according to a difference between abscissas of theextracted corner point coordinates; the determining a first height ofthe first rectangular calibration frame according to the first cornerpoint coordinates specifically comprises: extracting any two cornerpoint coordinates in the first corner point coordinates having a sameabscissa, and determining the first height according to a differencebetween ordinates of the extracted corner point coordinates; and thedetermining a second height of the second rectangular calibration frameaccording to the second corner point coordinates specifically comprises:extracting any two corner point coordinates in the second corner pointcoordinates having a same abscissa, and determining the second heightaccording to a difference between ordinates of the extracted cornerpoint coordinates.
 6. The method according to claim 3, wherein the sizecalibration parameter comprises a calibration width and a calibrationheight, and the position calibration parameter comprises origincalibration coordinates; and the determining the size calibrationparameter and the position calibration parameter of the thermal imageaccording to the zoom scale specifically comprises: acquiring a firstinitial width, a first initial height and first coordinates of a firstpositioning point that are of the thermal image; determining secondcoordinates of a second positioning point in the visible light imagecorresponding to the first positioning point; determining thecalibration width according to the zoom scale and the first initialwidth; determining the calibration height according to the zoom scaleand the first initial height; and determining the origin calibrationcoordinates according to the zoom scale, the first coordinates and thesecond coordinates.
 7. The method according to claim 6, wherein thefirst coordinates are any of the first upper-left corner pointcoordinates, the first lower-left corner point coordinates, the firstupper-right corner point coordinates or the first lower-right cornerpoint coordinates; and the second coordinates are any of the secondupper-left corner point coordinates, the second lower-left corner pointcoordinates, the second upper-right corner point coordinates or thesecond lower-right corner point coordinates.
 8. The method according toclaim 6, wherein after the step of fusing the to-be-fused images, themethod further comprises: determining a coincident region of a thermalimage region and a visible light region in the to-be-fused images; andcropping regions in the to-be-fused images other than the coincidentregion.
 9. The method according to claim 8, wherein the determiningcoincident region of a thermal image region and a visible light regionin the to-be-fused images specifically comprises: determining thirdcorner point coordinates of the thermal image region, wherein the thirdcorner point coordinates comprise third upper-left corner pointcoordinates, third lower-left corner point coordinates, thirdupper-right corner point coordinates and third lower-right corner pointcoordinates; determining fourth corner point coordinates of the visiblelight region, wherein the fourth corner point coordinates comprisefourth upper-left corner point coordinates, fourth lower-left cornerpoint coordinates, fourth upper-right corner point coordinates andfourth lower-right corner point coordinates; determining coordinates ofa first corner point and a second corner point of the coincident regionaccording to the third corner point coordinates and the fourth cornerpoint coordinates, wherein the first corner point and the second cornerpoint are diagonal corner points; and determining coordinates of a thirdcorner point and a fourth corner point of the coincident regionaccording to the coordinates of the first corner point and the secondcorner point.
 10. The method according to claim 9, wherein an origin ofthe thermal image is any of an upper-left corner point, a lower-leftcorner point, an upper-right corner point or a lower-right corner pointof the thermal image; and the determining third corner point coordinatesof the thermal image region specifically comprises: determining thethird corner point coordinates of the thermal image region according tothe calibration width, the calibration height and the origin calibrationcoordinates.
 11. The method according to claim 9, wherein an origin ofthe visible light image is any of an upper-left corner point, alower-left corner point, an upper-right corner point or a lower-rightcorner point of the visible light image; and the determining fourthcorner point coordinates of the visible light region specificallycomprises: acquiring a second initial width, a second initial height andorigin coordinates of the visible light image; and determining thefourth corner point coordinates of the visible light region according tothe second initial width, the second initial height and the origincoordinates.
 12. The method according to claim 9, wherein the firstcorner point and the second corner point are respectively a lower-leftcorner point and an upper-right corner point of the coincident region;and formulas for determining the coordinates of the first corner pointand the second corner point of the coincident region are as follows:$\left\{ {\begin{matrix}{{x_{1}^{\prime} \leq x_{1}^{''} \leq x_{2}^{\prime}},{x_{1} = x_{1}^{''}}} \\{{x_{1}^{''} < x_{1}^{\prime}},{x_{1} = x_{1}^{\prime}}}\end{matrix},\left\{ \begin{matrix}{{y_{1}^{\prime} \leq y_{1}^{''} \leq y_{2}^{\prime}},{y_{1} = y_{1}^{''}}} \\{{y_{1}^{''} < y_{1}^{\prime}},{y_{1} = y_{1}^{\prime}}}\end{matrix} \right.} \right.$ $\left\{ {\begin{matrix}{{x_{2}^{''} \leq x_{2}^{\prime}},{x_{2} = x_{2}^{''}}} \\{{x_{2}^{''} > x_{2}^{\prime}},{x_{2} = x_{2}^{\prime}}}\end{matrix},\left\{ \begin{matrix}{{y_{2}^{''} \leq y_{2}^{\prime}},{y_{2} = y_{2}^{''}}} \\{{y_{2}^{''} > y_{2}^{\prime}},{y_{2} = y_{2}^{\prime}}}\end{matrix} \right.} \right.$ wherein (x₁,y₁) and (x₂,y₂) arerespectively the coordinates of the first corner point and the secondcorner point; (x₁″,y₂″) (x₁″,y₁″), (x₂″,y₂″) and (x₂″,y₁″) and arerespectively the third upper-left corner point coordinates, the thirdlower-left corner point coordinates, the third upper-right corner pointcoordinates and the third lower-right corner point coordinates; and(x₁′,y₂′), (x₁′,y₁′), (x₂′,y₂′) and (x₂′,y₁′) are respectively thefourth upper-left corner point coordinates, the fourth lower-left cornerpoint coordinates, the fourth upper-right corner point coordinates andthe fourth lower-right corner point coordinates.
 13. An image fusionapparatus, applicable to a bifocal camera, the bifocal camera comprisinga thermal imaging lens and a visible light lens, wherein the apparatuscomprises: an acquisition module, configured to acquire a thermal imagecaptured by the thermal imaging lens and a visible light image capturedby the visible light lens; a determination module, configured to:determine a first focal length when the thermal imaging lens capturesthe thermal image and a second focal length when the visible light lenscaptures the visible light image; and determine a size calibrationparameter and a position calibration parameter of the thermal imageaccording to the first focal length and the second focal length; anadjustment module, configured to adjust a size of the thermal imageaccording to the size calibration parameter, and move the adjustedthermal image to the visible light image according to the positioncalibration parameter for registration with the visible light image, toobtain to-be-fused images; and a processing module, configured to fusethe to-be-fused images to generate a bifocal fused image.
 14. A bifocalcamera, comprising: a thermal imaging lens, configured to capture athermal image; a visible light lens, configured to capture a visiblelight image; at least one processor; and a memory, communicativelyconnected to the at least one processor, wherein the memory storesinstructions executable by the at least one processor, the instructions,when executed by the at least one processor, causing the at least oneprocessor to: acquiring a thermal image captured by the thermal imaginglens and a visible light image captured by the visible light lens;determining a first focal length when the thermal imaging lens capturesthe thermal image and a second focal length when the visible light lenscaptures the visible light image; determining a size calibrationparameter and a position calibration parameter of the thermal imageaccording to the first focal length and the second focal length;adjusting a size of the thermal image according to the size calibrationparameter, and moving the adjusted thermal image to the visible lightimage according to the position calibration parameter for registrationwith the visible light image, to obtain to-be-fused images; and fusingthe to-be-fused images to generate a bifocal fused image.
 15. The cameraaccording to claim 14, wherein the at least one processor are furtherconfigured to: determine a zoom scale of the thermal image according tothe first focal length and the second focal length; and determine thesize calibration parameter and the position calibration parameter of thethermal image according to the zoom scale.
 16. The camera according toclaim 15, wherein the thermal image comprises a first rectangularcalibration frame, and the visible light image comprises a secondrectangular calibration frame; and the at least one processor arefurther configured to: determine first corner point coordinates of thefirst rectangular calibration frame according to the first focal length,wherein the first corner point coordinates comprise first upper-leftcorner point coordinates, first lower-left corner point coordinates,first upper-right corner point coordinates and first lower-right cornerpoint coordinates; determine second corner point coordinates of thesecond rectangular calibration frame according to the second focallength, wherein the second corner point coordinates comprise secondupper-left corner point coordinates, second lower-left corner pointcoordinates, second upper-right corner point coordinates and secondlower-right corner point coordinates; and determine the zoom scale ofthe thermal image according to the first corner point coordinates andthe second corner point coordinates.
 17. The camera according to claim16, wherein the at least one processor are further configured to:determine a first height of the first rectangular calibration frameaccording to the first corner point coordinates; determine a secondheight of the second rectangular calibration frame according to thesecond corner point coordinates; and determine the zoom scale accordingto the first height and the second height.
 18. The camera according toclaim 17, wherein the at least one processor are further configured to:extract any two corner point coordinates in the first corner pointcoordinates having a same ordinate, and determining the first widthaccording to a difference between abscissas of the extracted cornerpoint coordinates; extract any two corner point coordinates in thesecond corner point coordinates having a same ordinate, and determiningthe second width according to a difference between abscissas of theextracted corner point coordinates; extract any two corner pointcoordinates in the first corner point coordinates having a sameabscissa, and determining the first height according to a differencebetween ordinates of the extracted corner point coordinates; and extractany two corner point coordinates in the second corner point coordinateshaving a same abscissa, and determining the second height according to adifference between ordinates of the extracted corner point coordinates.19. The camera according to claim 16, wherein the size calibrationparameter comprises a calibration width and a calibration height, andthe position calibration parameter comprises origin calibrationcoordinates; and the at least one processor are further configured to:acquire a first initial width, a first initial height and firstcoordinates of a first positioning point that are of the thermal image;determine second coordinates of a second positioning point in thevisible light image corresponding to the first positioning point;determine the calibration width according to the zoom scale and thefirst initial width; determine the calibration height according to thezoom scale and the first initial height; and determine the origincalibration coordinates according to the zoom scale, the firstcoordinates and the second coordinates.
 20. The camera according toclaim 19, wherein the first coordinates are any of the first upper-leftcorner point coordinates, the first lower-left corner point coordinates,the first upper-right corner point coordinates or the first lower-rightcorner point coordinates; and the second coordinates are any of thesecond upper-left corner point coordinates, the second lower-left cornerpoint coordinates, the second upper-right corner point coordinates orthe second lower-right corner point coordinates.
 21. The cameraaccording to claim 19, wherein after the step of fusing the to-be-fusedimages, the at least one processor are further configured to: determinea coincident region of a thermal image region and a visible light regionin the to-be-fused images; and crop regions in the to-be-fused imagesother than the coincident region.
 22. The camera according to claim 21,wherein the at least one processor are further configured to: determinethird corner point coordinates of the thermal image region, wherein thethird corner point coordinates comprise third upper-left corner pointcoordinates, third lower-left corner point coordinates, thirdupper-right corner point coordinates and third lower-right corner pointcoordinates; determine fourth corner point coordinates of the visiblelight region, wherein the fourth corner point coordinates comprisefourth upper-left corner point coordinates, fourth lower-left cornerpoint coordinates, fourth upper-right corner point coordinates andfourth lower-right corner point coordinates; determine coordinates of afirst corner point and a second corner point of the coincident regionaccording to the third corner point coordinates and the fourth cornerpoint coordinates, wherein the first corner point and the second cornerpoint are diagonal corner points; and determine coordinates of a thirdcorner point and a fourth corner point of the coincident regionaccording to the coordinates of the first corner point and the secondcorner point.
 23. The camera according to claim 22, wherein an origin ofthe thermal image is any of an upper-left corner point, a lower-leftcorner point, an upper-right corner point or a lower-right corner pointof the thermal image; and the at least one processor are furtherconfigured to: determine the third corner point coordinates of thethermal image region according to the calibration width, the calibrationheight and the origin calibration coordinates.
 24. The camera accordingto claim 22, wherein an origin of the visible light image is any of anupper-left corner point, a lower-left corner point, an upper-rightcorner point or a lower-right corner point of the visible light image;and the at least one processor are further configured to: acquire asecond initial width, a second initial height and origin coordinates ofthe visible light image; and determine the fourth corner pointcoordinates of the visible light region according to the second initialwidth, the second initial height and the origin coordinates.
 25. Thecamera according to claim 22, wherein the first corner point and thesecond corner point are respectively a lower-left corner point and anupper-right corner point of the coincident region; and formulas fordetermining the coordinates of the first corner point and the secondcorner point of the coincident region are as follows:$\left\{ {\begin{matrix}{{x_{1}^{\prime} \leq x_{1}^{''} \leq x_{2}^{\prime}},{x_{1} = x_{1}^{''}}} \\{{x_{1}^{''} < x_{1}^{\prime}},{x_{1} = x_{1}^{\prime}}}\end{matrix},\left\{ \begin{matrix}{{y_{1}^{\prime} \leq y_{1}^{''} \leq y_{2}^{\prime}},{y_{1} = y_{1}^{''}}} \\{{y_{1}^{''} < y_{1}^{\prime}},{y_{1} = y_{1}^{\prime}}}\end{matrix} \right.} \right.$ $\left\{ {\begin{matrix}{{x_{2}^{''} \leq x_{2}^{\prime}},{x_{2} = x_{2}^{''}}} \\{{x_{2}^{''} > x_{2}^{\prime}},{x_{2} = x_{2}^{\prime}}}\end{matrix},\left\{ \begin{matrix}{{y_{2}^{''} \leq y_{2}^{\prime}},{y_{2} = y_{2}^{''}}} \\{{y_{2}^{''} > y_{2}^{\prime}},{y_{2} = y_{2}^{\prime}}}\end{matrix} \right.} \right.$ wherein (x₁,y₁) and (x₂,y₂) arerespectively the coordinates of the first corner point and the secondcorner point; (x₁″,y₂″) (x₁″,y₁″), (x₂″,y₂″) and (x₂″,y₁″) arerespectively the third upper-left corner point coordinates, the thirdlower-left corner point coordinates, the third upper-right corner pointcoordinates and the third lower-right corner point coordinates; and(x₁′,y₂′), (x₁′,y₁′), (x₂′,y₂′) and (x₂′,y₁′) are respectively thefourth upper-left corner point coordinates, the fourth lower-left cornerpoint coordinates, the fourth upper-right corner point coordinates andthe fourth lower-right corner point coordinates.